In contrast to fully supervised methods using pixel-wise mask labels, box-supervised instance segmentation takes advantage of simple box annotations, which has recently attracted increasing research attention. This paper presents a novel single-shot instance segmentation approach, namely Box2Mask, which integrates the classical level-set evolution model into deep neural network learning to achieve accurate mask prediction with only bounding box supervision. Specifically, both the input image and its deep features are employed to evolve the level-set curves implicitly, and a local consistency module based on a pixel affinity kernel is used to mine the local context and spatial relations. Two types of single-stage frameworks, i.e., CNN-based and transformer-based frameworks, are developed to empower the level-set evolution for box-supervised instance segmentation, and each framework consists of three essential components: instance-aware decoder, box-level matching assignment and level-set evolution. By minimizing the level-set energy function, the mask map of each instance can be iteratively optimized within its bounding box annotation. The experimental results on five challenging testbeds, covering general scenes, remote sensing, medical and scene text images, demonstrate the outstanding performance of our proposed Box2Mask approach for box-supervised instance segmentation. In particular, with the Swin-Transformer large backbone, our Box2Mask obtains 42.4% mask AP on COCO, which is on par with the recently developed fully mask-supervised methods. The code is available at: https://github.com/LiWentomng/boxlevelset.

与使用像素面罩标签的完全监督的方法相反，盒子监督实例细分利用了简单的盒子注释，该盒子注释最近吸引了许多研究注意力。在本文中，我们提出了一种新颖的单弹盒监督实例分割方法，该方法将经典级别设置模型与深度神经网络精致整合在一起。具体而言，我们提出的方法迭代地通过端到端的方式通过基于Chan-Vese的连续能量功能来学习一系列级别集。一个简单的掩码监督的SOLOV2模型可供选择，以预测实例感知的掩码映射为每个实例的级别设置。输入图像及其深度特征都被用作输入数据来发展级别集曲线，其中使用框投影函数来获得初始边界。通过最大程度地减少完全可分化的能量函数，在其相应的边界框注释中迭代优化了每个实例的级别设置。在四个具有挑战性的基准上的实验结果表明，在各种情况下，我们提出的强大实例分割方法的领先表现。该代码可在以下网址获得：https：//github.com/liwentomng/boxlevelset。

利用Stylegan的表现力及其分离的潜在代码，现有方法可以实现对不同视觉属性的现实编辑，例如年龄和面部图像的性别。出现了一个有趣而又具有挑战性的问题：生成模型能否针对他们博学的先验进行反事实编辑？由于自然数据集中缺乏反事实样本，我们以文本驱动的方式研究了这个问题，并具有对比语言图像预言（剪辑），这些（剪辑）甚至可以为各种反事实概念提供丰富的语义知识。与内域操作不同，反事实操作需要更全面地剥削夹包含的语义知识，以及对编辑方向的更微妙的处理，以避免被卡在局部最低或不需要的编辑中。为此，我们设计了一种新颖的对比损失，该损失利用了预定义的夹子空间方向，从不同的角度将编辑指向所需的方向。此外，我们设计了一个简单而有效的方案，该方案将（目标文本）明确映射到潜在空间，并将其与潜在代码融合在一起，以进行有效的潜在代码优化和准确的编辑。广泛的实验表明，我们的设计在乘坐各种反事实概念的目标文本驾驶时，可以实现准确，现实的编辑。

本文介绍了DCT-NET，这是一种新颖的图像翻译体系结构，可用于几张肖像风格。给定有限的样式示例（$ \ sim $ 100），新的体系结构可以产生高质量的样式转移结果，具有先进的能力，可以合成高保真内容和强大的一般性来处理复杂的场景（例如，遮挡和配件）。此外，它可以通过一个由部分观察（即风格化的头）训练的优雅评估网络启用全身图像翻译。几乎没有基于学习的样式转移是具有挑战性的，因为由于仅由少数几个培训示例形成的偏见分布，学到的模型很容易在目标域中过度拟合。本文旨在通过采用“首先校准，稍后翻译”的关键思想来应对挑战，并以本地注重的翻译探索增强的全球结构。具体而言，所提出的DCT-NET由三个模块组成：一个内容适配器从源照片借用功能的先验来校准目标样本的内容分布；使用仿射变换来释放空间语义约束的几何扩展模块；以及通过校准分布产生的样品的质地翻译模块学习细粒的转换。实验结果证明了所提出的方法在头部风格化方面具有优势及其对具有自适应变形的完整图像翻译的有效性。

关于驾驶场景图像的语义细分对于自动驾驶至关重要。尽管在白天图像上已经实现了令人鼓舞的性能，但由于暴露不足和缺乏标记的数据，夜间图像的性能不那么令人满意。为了解决这些问题，我们提出了一个称为双图像自动学习过滤器（拨号过滤器）的附加模块，以改善夜间驾驶条件下的语义分割，旨在利用不同照明下驾驶场景图像的内在特征。拨盘滤波器由两个部分组成，包括图像自适应处理模块（IAPM）和可学习的引导过滤器（LGF）。使用拨号过滤器，我们设计了无监督和有监督的框架，用于夜间驾驶场景细分，可以以端到端的方式进行培训。具体而言，IAPM模块由一个带有一组可区分图像过滤器的小型卷积神经网络组成，可以自适应地增强每个图像，以更好地相对于不同的照明。 LGF用于增强分割网络的输出以获得最终的分割结果。拨号过滤器轻巧有效，可以在白天和夜间图像中轻松应用它们。我们的实验表明，Dail过滤器可以显着改善ACDC_Night和Nightcity数据集的监督细分性能，而它展示了有关无监督的夜间夜间语义细分的最新性能，在黑暗的苏黎世和夜间驾驶测试床上。

A recent study has shown a phenomenon called neural collapse in that the within-class means of features and the classifier weight vectors converge to the vertices of a simplex equiangular tight frame at the terminal phase of training for classification. In this paper, we explore the corresponding structures of the last-layer feature centers and classifiers in semantic segmentation. Based on our empirical and theoretical analysis, we point out that semantic segmentation naturally brings contextual correlation and imbalanced distribution among classes, which breaks the equiangular and maximally separated structure of neural collapse for both feature centers and classifiers. However, such a symmetric structure is beneficial to discrimination for the minor classes. To preserve these advantages, we introduce a regularizer on feature centers to encourage the network to learn features closer to the appealing structure in imbalanced semantic segmentation. Experimental results show that our method can bring significant improvements on both 2D and 3D semantic segmentation benchmarks. Moreover, our method ranks 1st and sets a new record (+6.8% mIoU) on the ScanNet200 test leaderboard. Code will be available at https://github.com/dvlab-research/Imbalanced-Learning.

When using LiDAR semantic segmentation models for safety-critical applications such as autonomous driving, it is essential to understand and improve their robustness with respect to a large range of LiDAR corruptions. In this paper, we aim to comprehensively analyze the robustness of LiDAR semantic segmentation models under various corruptions. To rigorously evaluate the robustness and generalizability of current approaches, we propose a new benchmark called SemanticKITTI-C, which features 16 out-of-domain LiDAR corruptions in three groups, namely adverse weather, measurement noise and cross-device discrepancy. Then, we systematically investigate 11 LiDAR semantic segmentation models, especially spanning different input representations (e.g., point clouds, voxels, projected images, and etc.), network architectures and training schemes. Through this study, we obtain two insights: 1) We find out that the input representation plays a crucial role in robustness. Specifically, under specific corruptions, different representations perform variously. 2) Although state-of-the-art methods on LiDAR semantic segmentation achieve promising results on clean data, they are less robust when dealing with noisy data. Finally, based on the above observations, we design a robust LiDAR segmentation model (RLSeg) which greatly boosts the robustness with simple but effective modifications. It is promising that our benchmark, comprehensive analysis, and observations can boost future research in robust LiDAR semantic segmentation for safety-critical applications.

Denoising Diffusion Probabilistic Models (DDPMs) are emerging in text-to-speech (TTS) synthesis because of their strong capability of generating high-fidelity samples. However, their iterative refinement process in high-dimensional data space results in slow inference speed, which restricts their application in real-time systems. Previous works have explored speeding up by minimizing the number of inference steps but at the cost of sample quality. In this work, to improve the inference speed for DDPM-based TTS model while achieving high sample quality, we propose ResGrad, a lightweight diffusion model which learns to refine the output spectrogram of an existing TTS model (e.g., FastSpeech 2) by predicting the residual between the model output and the corresponding ground-truth speech. ResGrad has several advantages: 1) Compare with other acceleration methods for DDPM which need to synthesize speech from scratch, ResGrad reduces the complexity of task by changing the generation target from ground-truth mel-spectrogram to the residual, resulting into a more lightweight model and thus a smaller real-time factor. 2) ResGrad is employed in the inference process of the existing TTS model in a plug-and-play way, without re-training this model. We verify ResGrad on the single-speaker dataset LJSpeech and two more challenging datasets with multiple speakers (LibriTTS) and high sampling rate (VCTK). Experimental results show that in comparison with other speed-up methods of DDPMs: 1) ResGrad achieves better sample quality with the same inference speed measured by real-time factor; 2) with similar speech quality, ResGrad synthesizes speech faster than baseline methods by more than 10 times. Audio samples are available at https://resgrad1.github.io/.

Crowdsourcing, in which human intelligence and productivity is dynamically mobilized to tackle tasks too complex for automation alone to handle, has grown to be an important research topic and inspired new businesses (e.g., Uber, Airbnb). Over the years, crowdsourcing has morphed from providing a platform where workers and tasks can be matched up manually into one which leverages data-driven algorithmic management approaches powered by artificial intelligence (AI) to achieve increasingly sophisticated optimization objectives. In this paper, we provide a survey presenting a unique systematic overview on how AI can empower crowdsourcing - which we refer to as AI-Empowered Crowdsourcing(AIEC). We propose a taxonomy which divides algorithmic crowdsourcing into three major areas: 1) task delegation, 2) motivating workers, and 3) quality control, focusing on the major objectives which need to be accomplished. We discuss the limitations and insights, and curate the challenges of doing research in each of these areas to highlight promising future research directions.

Fine-grained classification and counting of bone marrow erythroid cells are vital for evaluating the health status and formulating therapeutic schedules for leukemia or hematopathy. Due to the subtle visual differences between different types of erythroid cells, it is challenging to apply existing image-based deep learning models for fine-grained erythroid cell classification. Moreover, there is no large open-source datasets on erythroid cells to support the model training. In this paper, we introduce BMEC (Bone Morrow Erythroid Cells), the first large fine-grained image dataset of erythroid cells, to facilitate more deep learning research on erythroid cells. BMEC contains 5,666 images of individual erythroid cells, each of which is extracted from the bone marrow erythroid cell smears and professionally annotated to one of the four types of erythroid cells. To distinguish the erythroid cells, one key indicator is the cell shape which is closely related to the cell growth and maturation. Therefore, we design a novel shape-aware image classification network for fine-grained erythroid cell classification. The shape feature is extracted from the shape mask image and aggregated to the raw image feature with a shape attention module. With the shape-attended image feature, our network achieved superior classification performance (81.12\% top-1 accuracy) on the BMEC dataset comparing to the baseline methods. Ablation studies also demonstrate the effectiveness of incorporating the shape information for the fine-grained cell classification. To further verify the generalizability of our method, we tested our network on two additional public white blood cells (WBC) datasets and the results show our shape-aware method can generally outperform recent state-of-the-art works on classifying the WBC. The code and BMEC dataset can be found on https://github.com/wangye8899/BMEC.